Digital Image Search System And Method

ABSTRACT

A method and system for matching an unknown facial image of an individual with an image of an unknown twin using facial recognition techniques and human perception is disclosed herein. The invention provides a internet hosted system to find, compare, contrast and identify similar characteristics among two or more individuals using a digital camera, cellular telephone camera, wireless device for the purpose of returning information regarding similar faces to the user The system features classification of unknown facial images from a variety of internet accessible sources, including mobile phones, wireless camera-enabled devices, images obtained from digital cameras or scanners that are uploaded from PCs, third-party applications and databases. The method and system uses human perception techniques to weight the feature vectors.

CROSS REFERENCES TO RELATED APPLICATIONS

The Present Application is a continuation application of U.S. patentapplication Ser. No. 12/573,129, filed on Oct. 4, 2009, which is acontinuation application of U.S. patent application Ser. No. 12/198,887,filed Aug. 27, 2008, now U.S. Pat. No. 7,599,527, which claims priorityto U.S. Provisional Patent No. 60/968,326, filed on Aug. 28, 2007, nowabandoned, and is a continuation-in-part application of U.S. patentapplication Ser. No. 11/534,667, filed on Sep. 24, 2006, now U.S. Pat.No. 7,450,740, which claims priority to U.S. Provisional PatentApplication No. 60/721,226, filed Sep, 28, 2005, now abandoned.

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

Not Applicable

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and system for classificationof digital facial images received over wireless digital networks or theInternet and retrieval of information associated with a classifiedimage.

2. Description of the Related Art

Classification of facial images using feature recognition software iscurrently used by various government agencies such as the Department ofHomeland Security (DHS) and the Department of Motor Vehicles (DMV) fordetecting terrorists, detecting suspected cases of identity fraud,automating border and passport control, and correcting mistakes in theirrespective facial image databases. Facial images stored in the DMV orDHS are digitized and stored in centralized databases, along withassociated information on the person. Examples of companies that providebiometric facial recognition software include Cross Match Technologies,Cognitec, Cogent Systems, and Iridian Technologies; of these, Cognitecalso provides a kiosk for digitally capturing images of people forstorage into their software.

Your face is an important part of who you are and how people identifyyou. Imagine how hard it would be to recognize an individual if allfaces looked the same. Except in the case of identical twins, the faceis arguably a person's most unique physical characteristic. While humanshave had the innate ability to recognize and distinguish different facesfor millions of years, computers are just now catching up.

Visionics, a company based in New Jersey, is one of many developers offacial recognition technology. The twist to its particular software,FACEIT, is that it can pick someone's face out of a crowd, extract thatface from the rest of the scene and compare it to a database full ofstored images. In order for this software to work, it has to know what abasic face looks like. Facial recognition software is based on theability to first recognize faces, which is a technological feat initself, and then measure the various features of each face.

If you look in the mirror, you can see that your face has certaindistinguishable landmarks. These are the peaks and valleys that make upthe different facial features. Visionics defines these landmarks asnodal points. There are about 80 nodal points on a human face. A few ofthe nodal points that are measured by the FACEIT software: distancebetween eyes; width of nose; depth of eye sockets; cheekbones; Jaw line;and chin. These nodal points are measured to create a numerical codethat represents the face in a database. This code is referred to as afaceprint and only fourteen to twenty-two nodal points are necessary forthe FACEIT software to complete the recognition process.

Facial recognition methods may vary, but they generally involve a seriesof steps that serve to capture, analyze and compare your face to adatabase of stored images. The basic process that is used by the FACEITsoftware to capture and compare images is set forth below and involvesDetection, Alignment, Normalization, Representation, and Matching. Toidentify someone, facial recognition software compares newly capturedimages to databases of stored images to see if that person is in thedatabase.

Detection is when the system is attached to a video surveillance system,the recognition software searches the field of view of a video camerafor faces. If there is a face in the view, it is detected within afraction of a second. A multi-scale algorithm is used to search forfaces in low resolution. The system switches to a high-resolution searchonly after a head-like shape is detected.

Alignment is when a face is detected, the system determines the head'sposition, size and pose. A face needs to be turned at least thirty-fivedegrees toward the camera for the system to register the face.

Normalization is when the image of the head is scaled and rotated sothat the head can be registered and mapped into an appropriate size andpose. Normalization is performed regardless of the head's location anddistance from the camera. Light does not impact the normalizationprocess.

Representation is when the system translates the facial data into aunique code. This coding process allows for easier comparison of thenewly acquired facial data to stored facial data.

Matching is when the newly acquired facial data is compared to thestored data and linked to at least one stored facial representation.

The heart of the FACEIT facial recognition system is the Local FeatureAnalysis (LFA) algorithm. This is the mathematical technique the systemuses to encode faces. The system maps the face and creates thefaceprint. Once the system has stored a faceprint, it can compare it tothe thousands or millions of faceprints stored in a database. Eachfaceprint is stored as an 84-byte file.

One of the first patents related to facial recognition technology isRothfjell, U.S. Pat. No. 3,805,238 for a Method For IdentifyingIndividuals using Selected Characteristics Body Curves. Rothfjellteaches an identification system in which major features (e.g. the shapeof a person's nose in profile) are extracted from an image and stored.The stored features are subsequently retrieved and overlaid on a currentimage of the person to verify identity.

Another early facial recognition patent is Himmel, U.S. Pat. No.4,020,463 for an Apparatus And A Method For Storage And Retrieval OfImage Patterns. Himmel discloses digitizing a scanned image into binarydata which is then compressed and then a sequence of coordinates andvector values are generated which describe the skeletonized image. Thecoordinates and vector values allow for compact storage of the image andfacilitate regeneration of the image.

Yet another is Gotanda, U.S. Pat. No. 4,712,103 for a Door Lock ControlSystem. Gotanda teaches, inter alia, storing a digitized facial image ina non-volatile ROM on a key, and retrieving that image for comparisonwith a current image of the person at the time he/she request access toa secured area. Gotanda describes the use of image compression, by asmuch as a factor of four, to reduce the amount of data storage capacityneeded by the ROM that is located on the key.

Yet another is Lu, U.S. Pat. No. 4,858,000. Lu teaches an imagerecognition system and method for identifying ones of a predeterminedset of individuals, each of whom has a digital representation of his orher face stored in a defined memory space.

Yet another is Tal, U.S. Pat. No. 4,975,969. Tal teaches an imagerecognition system and method in which ratios of facial parameters(which Tal defines a distances between definable points on facialfeatures such as a nose, mouth, eyebrow etc.) are measured from a facialimage and are used to characterize the individual. Tal, like Lu in U.S.Pat. No. 4,858,000, uses a binary image to find facial features.

Yet another is Lu, U.S. Pat. No. 5,031,228. Lu teaches an imagerecognition system and method for identifying ones of a predeterminedset of individuals, each of whom has a digital representation of his orher face stored in a defined memory space. Face identification data foreach of the predetermined individuals are also stored in a UniversalFace Model block that includes all the individual pattern images or facesignatures stored within the individual face library.

Still another is Burt, U.S. Pat. No. 5,053,603. Burt teaches an imagerecognition system using differences in facial features to distinguishone individual from another. Burt's system uniquely identifiesindividuals whose facial images and selected facial feature images havebeen learned by the system. Burt's system also “generically recognizes”humans and thus distinguishes between unknown humans and non-humanobjects by using a generic body shape template.

Still another is Turk et al., U.S. Pat. No. 5,164,992. Turk teaches theuse of an Eigenface methodology for recognizing and identifying membersof a television viewing audience. The Turk system is designed to observea group of people and identify each of the persons in the group toenable demographics to be incorporated in television ratingsdeterminations.

Still another is Deban et al., U.S. Pat. No. 5,386,103. Deban teachesthe use of an Eigenface methodology for encoding a reference face andstoring said reference face on a card or the like, then retrieving saidreference face and reconstructing it or automatically verifying it bycomparing it to a second face acquired at the point of verification.Deban teaches the use of this system in providing security for AutomaticTeller Machine (ATM) transactions, check cashing, credit card securityand secure facility access.

Yet another is Lu et al., U.S. Pat. No. 5,432,864. Lu teaches the use ofan Eigenface methodology for encoding a human facial image and storingit on an “escort memory” for later retrieval or automatic verification.Lu teaches a method and apparatus for employing human facial imageverification for financial transactions.

Technologies provided by wireless carriers and cellular phonemanufacturers enable the transmission of facial or object images betweenphones using Multimedia Messaging Services (MMS) as well as to theInternet over Email (Simple Mail Transfer Protocol, SMTP) and WirelessAccess Protocol (WAP). Examples of digital wireless devices capable ofcapturing and receiving images and text are camera phones provided byNokia, Motorola, LG, Ericsson, and others. Such phones are capable ofhandling images as JPEGs over MMS, Email, and WAP across many of thewireless carriers: Cingular, T-Mobile, (GSM/GPRS), and Verizon (CDMA)and others.

Neven, U.S. Patent Publication 2005/0185060, for an Image Base Inquirysystem For Search Engines For Mobile Telephones With Integrated Camera,discloses a system using a mobile telephone digital camera to send animage to a server that converts the image into symbolic information,such as plain text, and furnishes the user links associated with theimage which are provided by search engines.

Neven, et al., U.S. Patent Publication 2006/0012677, for an Image-BasedSearch Engine For Mobile Phones With Camera, discloses a system thattransmits an image of an object to a remote server which generates threeconfidence values and then only generates a recognition output from thethree confidence values, with nothing more. I

Adam et al., U.S. Patent Publication 2006/0050933, for a Single ImageBased Multi-Biometric System And Method which integrates face, skin andiris recognition to provide a biometric system.

Until recently, acquiring information about someone from a real-timeimage has always been the domain of science fiction novels. Recently,the government and large companies (such as casinos) have implementedface recognition systems to identify individuals from a real-time image.However, do to the costs and lack of a database these systems are notavailable to the individual member of the general public. Further, thepresent systems rely on the individual being present geographically andan image of the individual being provided on a predetermined databasesuch as government database of images of terrorists or a casino databaseof images of known “card cheaters.”

BRIEF SUMMARY OF THE INVENTION

The present invention provides a novel method and system for providingan individual an expedient, inexpensive and technologically easy meansfor determining if another individual looks like the individual,essentially determining if an unknown “twin” exists for the individual.

The invention classifies a person, or whom a person most looks like, bypreferably using a digital image captured by a wireless communicationdevice (preferably a mobile telephone) or from a personal computer (PC).The image may be in a JPEG, TIFF, GIF or other standard image format.Further, an analog image may be utilized if digitized. The image is sentto the wireless carrier and subsequently sent over the internet to animage classification server. Alternatively, the digital image may beuploaded to a PC from a digital camera or scanner and then sent to theimage classification server over the internet.

After an image is received by the image classification server, the imageis processed into a feature vector, which reduces the complexity of thedigital image data into a small set of variables that represent thefeatures of the image that are of interest for classification purposes.

The feature vector is compared against existing feature vectors in animage database to find the closest match. The image database preferablycontains one or more feature vectors for each target individual.

Once classified, an image of the best matching person, possiblymanipulated to emphasize matching characteristics, as well as meta-dataassociated with the person, sponsored information, similar product,inventory or advertisement is sent back to the user's PC or wirelesscommunication device.

A more detailed explanation of a preferred method of the invention is asfollows below. The user captures a digital image with a digital cameraenabled wireless communication device, such as a mobile telephone. Thecompressed digital image is sent to the wireless carrier as a multimediamessage (MMS), a short message service (“SMS”), an e-mail (Simple MailTransfer Protocol (“SMTP”)), or wireless application protocol (“WAP”)upload. The image is subsequently sent over the internet using HTTP ore-mail to an image classification server. Alternatively, the digitalimage may be uploaded to a PC from a digital camera, or scanner. Once onthe PC, the image can be transferred over the Internet to the imageclassification server as an e-mail attachment, or HTTP upload. The useris preferably the provider of the digital image for classification, andincludes, but is not limited to a physical person, machine, or softwareapplication.

After the image is received by the image classification server, afeature vector is generated for the image. A feature vector is a smallset of variables that represent the features of the image that are ofinterest for classification purposes. Creation and comparison offeatures vectors may be queued, and scaled across multiple machines.Alternatively, different feature vectors may be generated for the sameimage. Alternatively, the feature vectors of several images of the sameindividual may be combined into a single feature vector. The incomingimage, as well as associate features vectors, may be stored for laterprocessing, or added to the image database. For faces, possible featurevector variables are the distance between the eyes, the distance betweenthe center of the eyes, to the chin, the size, and shape of theeyebrows, the hair color, eye color, facial hair if any, and the like.

After the feature vector for an image is created, the feature vector iscompared against feature vectors in an image database to find theclosest match. Preferably, each image in the image database has afeature vector. Alternatively, feature vectors for the image databaseare created from a set of faces, typically eight or more digital imagesat slightly different angles for each individual. Since the targetindividual's feature vector may be generated from several images, anoptional second pass is made to find which of the individual images thatwere used to create the feature vector for the object best match theincoming image.

Once classified, the matching image's name and associated meta-data isretrieved from the database. Before the response is sent, thebest-matching image or incoming image may be further manipulated toemphasize the similarities between the two images. This imagemanipulation can be automated, or can be done interactively by the user.The matching image's name, meta-data, associated image, and a copy ofthe incoming image are then sent back to the user's wirelesscommunication device or PC, and also to a web page for the user.

One preferred aspect of the present invention is a method for matchingimages. The method includes acquiring a facial image of a human. Next,the facial image is transmitted from a sender to a server. Next, thefacial image is analyzed at the server to determine if the facial imageis acceptable. Next, the facial image is processed to create a processedimage. Next, the processed image is compared to a plurality of databaseprocessed images. Next, the processed image is matched to a databaseprocessed image of the plurality of database processed images to creatematched images. Next, a perception value of the matched images isdetermined at the server site. Then, the matched images and theperception value are transmitted to the sender.

Another aspect of the present invention is a method for matching animage of an individual to an image of an unknown “twin”, wherein in twinis defined as similar in appearance of facial features. The methodincludes wirelessly transmitting a digital facial image of an individualfrom a mobile communication device over a wireless network to an imageclassification server. Next, the digital facial image is processed atthe image classification server to create a primary feature vector forthe digital facial image. Next, the primary feature vector is comparedto a plurality of database feature vectors, with each of the pluralityof database feature vectors corresponding to a database processed image.Next, a database feature vector is selected that best matches theprimary feature vector to create matched images of the digital facialimage of the individual and a twin. Next, the matched images aretransmitted to the mobile communication device. The twin image can be animage that was sent into the image classification server and added tothe plurality of database feature vectors.

Yet another aspect of the present invention is a system for matching anunknown facial image of an individual with an image of a twin. Thesystem includes a mobile communication device, an image classificationserver and a wireless network. The mobile communication device includesmeans for generating a digital facial image of an individual and meansfor wireless transmitting the digital facial image. The imageclassification server has means for receiving the digital facial imagefrom the mobile communication device, means for analyzing the digitalfacial image, means for processing the digital facial image to generatea processed image, means for comparing the processed image to aplurality of database processed images, means for matching the processedimage to a database processed image of the plurality of databaseprocessed images to create matched images, means for determining aperception value of the matched images, and means for transmitting thematched images and the confidence value to the mobile communicationdevice. The wireless network allows for transmissions between the mobilecommunication device and the image classification server.

The processed image is preferably processed as a primary feature vectorand the plurality of database processed images is a plurality ofdatabase feature vectors. Comparing the processed image to a pluralityof database processed images preferably comprises comparing the primaryfeature vector to each of the plurality of database feature vectors. Theprimary feature vector and each of the plurality of database featurevectors are preferably based on a plurality of factors comprising facialexpression, hair style, hair color, facial pose, eye color, localfeature analysis, eigenfaces, principle component analysis, texture ofthe face, color of the face and facial hair.

The method preferably further comprises web crawling a plurality of Websites for images of individuals to process each of the images to add tothe databases of processed images with each of the images of thedatabases of processed images having a tag for linking to the Web sitepertaining to the image.

Having briefly described the present invention, the above and furtherobjects, features and advantages thereof will be recognized by thoseskilled in the pertinent art from the following detailed description ofthe invention when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a flow chart of a specific method of the present invention.

FIG. 2 is a flow chart of a general method of the present invention.

FIG. 3 is a schematic diagram of a system of the present invention.

FIG. 3A is a schematic representation of the image classification serverof the present invention.

FIG. 4 is image and table comparison of an individual's image and anunknown image.

FIG. 5 is a flow chart of a specific method of the present invention.

FIG. 6 is a flow chart of a specific method of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

A flow chart of a method is illustrated in FIG. 1. The method isgenerally designated 100 and commences with a facial image of individualbeing acquired at block 101. The facial image is acquired preferablyusing a digital camera of a wireless communication device such as awireless mobile telephone, personal digital assistant (“PDA”) or thelike. Alternatively, the facial image is acquired from a PC or the like.

At block 102, the facial image is transmitted over a network to an imageclassification server, preferably over a wireless network. The facialimage is preferably sent to a male or female designation site at theimage classification server. The facial image is subsequently sent overthe internet using HTTP or e-mail to the image classification server.The facial image, preferably a compressed digital facial image such as aJPEG image, is sent to a wireless carrier as a MMS, a SMS, a SMTP, orWAP upload. Alternatively, the facial image is uploaded to a PC from adigital camera, or scanner and then transferred over the internet to theimage classification server as an e-mail attachment, or HTTP upload.

At block 103, the facial image is analyzed at the image classificationsserver to determine if the facial image is of adequate quality to beprocessed for matching. Quality issues with the facial image include butare not limited to a poor pose angle, brightness, shading, eyes closed,sunglasses worn, obscured facial features, or the like. At block 104, animage determination is made concerning the quality of the image. Anegative image determination is made at block 105. At block 106, atransmission is sent to the sender informing then sender that the facialimage provided is inadequate and requesting that the sender provide anew facial image. The matching procedure for such a negative image maycontinue, and the matched images will be sent with an additionalstatement informing the sender that the image was of bad quality andthat a better match may be possible with a higher quality image.

At block 107, if the facial image is positive, then the facial image isprocessed at block 108. It should be noted that the facial image ispreviously unknown to the image classification and is the first timethat the facial image has been analyzed by the image classificationserver. Thus, the method of present invention involves processing anunknown image to find a match with facial images of other individuals,which is unlike typical facial recognition systems which involvematching an image of an individual with a known image of the individualin the database. At block 108, processing of the image preferablycomprises using an algorithm which includes a principle componentanalysis technique to process the face of the facial image into anaverage of a multitude of faces, otherwise known as the principlecomponent and a set of images that are the variance from the averageface image known as the additional components. Each is reconstructed bymultiplying the principal components and the additional componentsagainst a feature vector and adding the resulting images together. Theresulting image reconstructs the original face of the facial image.Processing of the facial image comprises factors such as facial hair,hair style, facial expression, the presence of accessories such assunglasses, hair color, eye color, and the like. Essentially a primaryfeature vector is created for the facial image.

At block 109, processed image or primary feature vector is compared to aplurality of database processed images preferably located at the imageclassification server. During the comparison, the primary feature vectoris compared a plurality of database feature vectors which represent theplurality of database processed images. The database preferably includesat least 100,000s of processed images, more preferably at least1,000,000 processed images, and most preferably from 100,000 processedimages to 10,000,000 processed images. Those skilled in the pertinentart will recognize that the database may contain any number of imageswithout departing from the scope and spirit of the present invention.The processed images preferably include multiple images of oneindividual, typically from two to twenty images, more preferably fromfour to ten images of a single individual in different poses, differentfacial expressions, different hair styles and the like. The database ofprocessed images preferably includes other individuals of the publicwhich have sent their image into the image classification server. Again,it should be noted that the facial image sent by the sender is anunknown image which is being best matched to a known image. The databaseof processed images preferably includes images acquired from socialnetworking Web sites, other publicly accessible Web sites, private Websites, and government Web sites. These images are preferably obtainedworking with the owners of the Web site or using a Web crawling orspider program to obtain images and information for processing intofeature vectors.

At block 110, the processed image undergoes raw matching of a smallplurality of database images with each having a feature vector valuethat is close to the value of the primary feature vector. At block 110a, the iterative processing of the raw matching is performed wherein thehuman perception of what is a good match is one of the primary factorsin creating the matched images. At block 111, preferably a perceptionvalue for the matched images is determined based on the feature vectorvalues. The perception value ranges from 0% to 100%, with 100% being anideal match. At block 111 a, the matches are sorted based on predictedhuman perception.

At block 112, the matched images and the perception value aretransmitted to the sender over a network as discussed above for theinitial transmission. The entire process preferably occurs within a timeperiod of sixty seconds, and most preferably within a time of tenseconds. The process may be delayed due to the wireless carrier, andnetwork carrier. In this manner, the sender will know which celebritythe facial image best matches. The output of the matched images and anyadditional text is preferably sent to the sender's wirelesscommunication device for instantaneous feedback of their inquiry ofwhich celebrity does the facial image look like. Further, the output isalso sent to a sender's web page on a web site hosted through the imageclassification server wherein the sender can control access to thesender's web page and modify the matched images and the additional text.Further, the output is sent to a voting site as discussed below.

At decision 113, the quality of the matched images is determined todecide if the matched images should be sent to voting site on the website. At block 115, the matched images are sent to the sender's wirelesscommunication device, the sender's web page on the web site for viewingby the sender and other viewers determined by the sender. At block 114,the matched images are sent to the voting site if of sufficient quality,preferably based on the perception value, to be voted upon by visitorsto the voting site.

In this manner, a statistical modeling element is added to the matchingprocess to better match images based on human perception as determinedby the scores for previously matched images on the voting site. In otherembodiments regression analysis or Bayesian analysis is utilized. Underthis alternative scenario, a Support Vector Machine, preferably ahigh-dimensional neural network, with two feature vectors of a match,along with average vote scores collected from viewers of the web sitewill be utilized to provide better matching of images. A more detailedexplanation of a Support Vector Machine is set forth in Cortes & Vapnik,Support Vector Networks, Machine Learning, 20, 1995, which is herebyincorporated by reference in its entirety. The previous voting patternsare implemented in a statistical model for the algorithm to capture thehuman perception element to better match images as perceived by humans.

A more general method of the present invention is illustrated in FIG. 2.The general method is designated 150. At block 151, an unknown imagefrom a wireless communication device such as a mobile telephone istransmitted from a sender to an image classification server over anetwork such as a wireless network with subsequent internettransmission. At block 152, the unknown image is processed to create aprimary feature vector such as discussed above. At block 153, theprimary feature vector value is compared to a plurality of databasefeature vectors. At block 154, a database feature vector that bestmatches the primary feature vector is selected to create matched images.At block 155, the matched images are transmitted to the sender, alongwith a confidence value and other information about the matching image.

A system of the present invention is illustrated in FIG. 3. The systemis generally designated 50. The system 50 preferably comprises awireless communication device 51, a wireless network 52, an imageclassification server 53 and a web site 55, not shown, which may beviewed on a computer 54 or alternate wireless communication device 54′with internet access. The wireless communication device preferablycomprises means for generating a digital facial image of an individualand means for wirelessly transmitting the digital facial image over awireless network. The image classification server 53 preferablycomprises means for analyzing the digital facial image, means forprocessing the digital facial image to generate a processed image, meansfor comparing the processed image to a plurality of database processedimages, means for matching the processed image to a database processedimage to create matched images, means for determining a perceptionvalue, means for applying a statistical model based on human perceptionas determined by user's votes of previous third party matched images,and means for transmitting the matched images and information to thewireless communication device.

The present invention preferably uses facial recognition softwarecommercially or publicly available such as the FACEIT brand softwarefrom IDENTIX, the FACEVACS brand software from COGNETIC, and others.Those skilled in the pertinent art will recognize that there are manyfacial recognition softwares, including those in the public domain, thatmay be used without departing from the scope and spirit of the presentinvention.

The operational components of the image classification server 53 areschematically shown in FIG. 3A. The image classification server 53preferably comprises an input module 62, transmission engine 63, inputfeed 64, feature vector database 65, sent images database 66, facialrecognition software 67, perception engine 68, output module 69 and theimage database 70. The input module 62 is further partitioned intowireless device inputs 62 a, e-mail inputs 62 b and HTTP (internet)inputs 62 c. The output module 69 is further partitioned into wirelessdevice outputs 69 a, a sender's web page output 69 b and a voting webpage output 69 c. The feature vector database 65 is the database ofprocessed images of the celebrities from which the previously unknownfacial image is matched with one of the processed images. The imagedatabase is a database of the actual images from social networking Websites, other publicly accessible Web sites, private Web sites, andgovernment Web sites which are sent as outputs for the matched images.The sent images database 66 is a database of all of the images sent infrom users/senders to be matched with the processed images. Theperception engine 68 imparts the human perception processing to thematching procedure.

As shown in FIG. 4, an unknown facial image 80 sent by an individual ismatched to an image 75 selected from the database of processed imagesusing a method of the present invention as set forth above. The tableprovides a comparison of the facial values for each of the images.

The present invention also preferably uses voting results to weighfeature vectors. In addition to using vote results to select which actorimages are good for enrollment, vote results can also be used to weighthe feature vector itself so that qualities of the image that areperceived by humans are more heavily weighted when searching for a goodmatch. Biometric security software (Cognitec, Identix, etc.) selects andweighs the features of an image in order to match an image of a personto another image of the same person and optimizing the vector to achievethis result. The feature vector can be made up of local facial features,or overall components of the face as determined by principle componentanalysis.

The use of human perception voting results in order to optimize thelook-a-likeness of a person to a different person can use used,regardless of the how the feature vectors are determined. In otherwords, the algorithm for determining the set of feature vectors thatbest represent a face can be augmented with a 2^(nd) algorithm whichtakes these feature vectors, typically represented as a vector offloating point numbers, and weighs the values in the vector so that thecharacteristics of the image that are based on human perception are usedmore heavily. A more detailed explanation of human perception for facialrecognition is provided in Myers, et al., U.S. Pat. No. 7,587,070, forImage Classification And Information Retrieval Over Wireless DigitalNetworks And The Internet, which is hereby incorporated by references inits entirety.

Statistical methods such as neural networks or support vector machines(SVMs) can be used to feed the source and actor feature vectors andpredict the human perception vote.

The feature vector from the source image and the feature vector from theactor image are feed into a neural network which is trained on the humanperception rating for the match. Given many matches and correspondingvotes, the neural network can weigh the input vector values, v1, v2,etc. and see which of these feature vector components are statisticallyrelevant to the determination of the human vote or rating.

Once trained, the Neural Network or SVM can predict whether a match isgood or not by using the feature vectors, determined from a separatealgorithm.

A method 400 for determining an unknown twin for an individual isillustrated in FIG. 5. At block 402, an image of an unknown individualis obtained, using preferably a digital camera, and sent to an imageclassification server. The individual is unknown to an imageclassification server. At block 4040, a plurality of feature vectors isgenerated for the image. At block 406, an engine searches an existingdatabase of feature vectors for images to match the image to an unknowntwin of the individual. At block 408, the matching twin is located. Atblock 410, the information is sent to the individual including matchingtwin images.

A method 500 for determining a twin from a social networking site isshown in FIG. 6. In this method, at block 502, a person acquires animage of an individual, even himself or herself. The first person maytake a digital image, for example, using a camera phone on a mobiletelephone. At block 504, the image is then sent to a server as discussedabove in reference to FIG. 1, where a plurality of feature vectors aregenerated for the image of the first person. At block 506, the serversearches publicly accessible websites, such as myspace.com,facebook.com, and other social networking websites, to find a matchingtwin for the first person. At block 508, a matching twin is located andinformation from the website for the unknown individual is captured.Such information may be a URL for the web page of the unknown person,personal information, and other available information. At block 510, theinformation is transmitted to the first person.

From the foregoing it is believed that those skilled in the pertinentart will recognize the meritorious advancement of this invention andwill readily understand that while the present invention has beendescribed in association with a preferred embodiment thereof, and otherembodiments illustrated in the accompanying drawings, numerous changesmodification and substitutions of equivalents may be made thereinwithout departing from the spirit and scope of this invention which isintended to be unlimited by the foregoing except as may appear in thefollowing appended claim. Therefore, the embodiments of the invention inwhich an exclusive property or privilege is claimed are defined in thefollowing appended claims.

We claim as our invention:
 1. A method for matching a twin of a firstimage of an individual with a second image of an individual, the methodcomprising: analyzing an unknown facial image to determine if theunknown facial image is acceptable; processing the unknown facial imageto create a processed image; comparing the processed image to aplurality of database processed images; matching the processed image toa database processed image of the plurality of database processed imagesto create matched twin images, wherein the database processed image is afacial image of a twin in appearance to the unknown facial image;sorting matches based on predicted human perception to generate a bestmatched images; and transmitting the best matched images.
 2. The methodaccording to claim 1 wherein the processed image is processed as aprimary feature vector and the plurality of database processed images isa plurality of database feature vectors, and wherein comparing theprocessed image to a plurality of database processed images comprisescomparing the primary feature vector to each of the plurality ofdatabase feature vectors.
 3. The method according to claim 2 whereinmatching the processed image to a database processed image of theplurality of database processed images to create matched twin imagescomprises selecting a database feature vector with a value that is mostsimilar to the value of the primary feature vector.
 4. The methodaccording to claim 2 wherein comparing the processed image to aplurality of database processed images further comprises applying astatistical model based on human perception as determined by user'svotes of previous third party matched images.
 5. The method according toclaim 2 wherein the primary feature vector and each of the plurality ofdatabase feature vectors are based on a plurality of factors comprisingfacial expression, hair style, hair color, facial pose, eye color,texture of the face, color of the face and facial hair.
 6. The methodaccording to claim 1 further comprising transmitting the unknown facialimage from a sender to a server.
 7. The method according to claim 1further comprising acquiring the unknown facial image by a digitalcamera and transmitting the unknown facial image from a computer overthe internet to the server, or acquiring the unknown facial image acamera of a mobile telephone and transmitting the unknown facial imagefrom the mobile telephone over a wireless network to the server.
 8. Themethod according to claim 7 wherein the unknown facial image is a JPEGimage transmitted as a MMS.
 9. The method according to claim 7 whereinthe plurality of best matched images and a perception value aretransmitted from the server over a wireless network to the mobiletelephone.
 10. The method according to claim 1 wherein the matched twinimages and a perception value are transmitted to a sender's web page ona web site.
 11. The method according to claim 1 wherein analyzing theunknown facial image at the server comprises determining if a pluralityof facial image factors are acceptable, the plurality of facial imagefactors comprising the lack of a facial image, the lack of eyes, unevenlighting, the brightness of the facial image, pose angle of the facialimage, relative size of the facial image and pixel strength of thefacial image.
 12. A method for matching images using a network, themethod comprising: transmitting a digital facial image over a networkfrom a first point of the network to a second point of the network; andtransmitting a best match twin images from the second point of thenetwork to the first point of the network wherein the best match twinimages are a primary feature vector of a processed digital facial imageof the digital facial image and a database feature vector whichcorresponds to a database processed image of a plurality of databaseprocessed images, wherein the primary feature vector has been comparedto the database feature vector.
 13. The method according to claim 12wherein the first point of the network is a mobile communication deviceand the second point of the network is an image classification server.14. A method of matching an unknown facial image of a first individualwith an image of another individual that is a twin in appearance to thefirst individual, the method comprising: processing an unknown facialimage to create a processed image; comparing the processed image to aplurality of database processed images; matching the processed image toa database processed image of the plurality of database processed imagesto create matched images, wherein the database processed image is afacial image of another individual from the another individual's webpage of a publicly available website, the web page containing personalinformation of the individual and a uniform resource locator for the webpage; and transmitting the database processed image, the personalinformation of the individual and the uniform resource locator for theweb page.
 15. The method according to claim 14 wherein the personalinformation of the another individual comprises at least one of theanother individual's name, address, telephone number, email address,age, school, friends, favorite entertainments, and favorite foods.